Requirement Checklist (AWS)

This checklist covers what you need in place before installing Yeedu on AWS: the command-line tools on the machine you install from, an AWS account with an access and secret key pair, an EKS cluster, an S3 bucket, an RDS PostgreSQL instance, an ECR registry, CloudWatch log groups, the open ports, and an EFS file system with its mount targets. Work through it in order. Each item names the defaults the Yeedu CloudFormation template uses where one exists.

Software Requirements

Ensure the following command-line tools (CLIs) are available on the machine you install from:

• kubectl
• helm
• helmfile
• aws (AWS CLI)

We don't pin specific versions in the Yeedu installer. Use kubectl and helm releases that are compatible with the Kubernetes version of your cluster, and a helmfile release built against that helm version.

note

yeedu-core-services.sh installs kubectl, Helm, the Helmfile CLI, psql, rabbitmqadmin and the Redis CLI on the machine if they're missing, adapting to the machine's architecture and OS.

Cloud Requirements

1. Amazon Web Services (AWS) account

Make sure to have an active AWS account.

2. AWS Access & Secret Key Pair

Create a AWS Access & Secret Key Pair with following permissions: Create EC2 machines, Read VPC, Storage Object Admin, Read and Write access to ECR, CloudWatch Log Writer

3. Elastic Kubernetes Services (EKS)

Configure Elastic Kubernetes Service with the following recommendations:

• Namespace: yeedu (Recommended)
• Cluster Version: a Kubernetes version that is still in standard support on EKS
• Node capacity: at least 8 vCPU and 16 GB RAM available to the Yeedu control-plane workloads (the Yeedu CloudFormation template sizes its control-plane host at "Min 8 core, 16 GB RAM", default c6a.2xlarge)

4. S3 Storage Bucket

Create an S3 Storage Bucket for storing configuration files and logs. Ensure the IAM access key pair from Step 2 has the necessary read and write permissions.

5. AWS RDS (PostgreSQL)

Deploy an AWS RDS (PostgreSQL) instance. The values below are the defaults we use in the Yeedu CloudFormation template (automation/aws/yeedu-platform.yaml):

• Instance class: db.m5.large
• Engine: PostgreSQL 15.x (>=15 and <16); template default 15.12
• Database Name: yeedu
• Allocated storage: 20 GB, gp3
• Backup retention: 7 days
• Multi-AZ: enabled

note

Size the instance class and storage up from these defaults to match your own workload. We don't publish a capacity model, so treat any larger sizing as a local decision.

6. Elastic Container Registry (ECR)

Create an Elastic Container Registry (ECR) for storing Docker images. Refer to setup-docker-registry section for a list of container registries to be created and instructions on uploading Yeedu images.

7. AWS CloudWatch

Set up a AWS CloudWatch to view machine logs, create log groups in CloudWatch. Ensure the IAM access key pair from Step 2 has the necessary write access.

8. Firewalls

Ensure the following ports are open:

Service Name	Port	Access From Workstation	Access From Apache Spark Cluster	Access From Yeedu Control Plane Server
REST API	8080	Yes	Yes	Yes
Jupyter Notebook	8888-9088	No	No	Yes
RabbitMQ UI	15672	Yes	No	No
Redis	6379	No	No	Yes
RabbitMQ	5672	No	Yes	Yes
PostgreSQL	5432	Yes	Yes	Yes
History-server	10000	Yes	No	Yes
Grafana	3000	Yes	Yes	Yes
Influx DB	8086	No	Yes	Yes
LDAP	389	No	No	Yes

We also use the following ports on the platform. They're defined in bin/yeedu-env.sh and, except where noted, the Yeedu CloudFormation template opens them on the control-plane host:

Service Name	Port	Environment variable / notes
Yeedu UI	5173	YEEDU_UI_PORT
Vault	8200	YEEDU_RESTAPI_VAULT_PORT
Vault UI	9200	YEEDU_RESTAPI_VAULT_UI_PORT (not opened by the CloudFormation template)
Yeedu Functions scheduler	5171	YEEDU_FUNCTIONS_SCHEDULER_PORT (not opened by the CloudFormation template)
Yeedu Functions proxy	5172	YEEDU_FUNCTIONS_PROXY_PORT
Cache Proxy	8765	YEEDU_CACHE_PROXY_PORT; used only when YEEDU_CACHE_PROXY_ENABLED is true
Scheduler (Airflow)	8081	YEEDU_SCHEDULER_PORT; used only when YEEDU_ENABLE_SCHEDULER is true
AssistantX	8000 / 8001	YEEDU_ASSISTANT_PORT / YEEDU_ASSISTANT_FASTAPI_PORT; used only when YEEDU_ASSISTANT_ENABLED is true
EFS (NFS)	2049	Required from the control-plane host to the EFS mount targets

Make sure to fulfill these prerequisites for a successful Yeedu installation.

EFS prerequisites

• Create an EFS file system and corresponding mount targets.
• EC2 instance and EFS mount targets should be in the same VPC or connected VPCs to facilitate network communication.
• Allow inbound traffic on port 2049 from EC2 instances to the EFS mount targets.
• Availability Zone : regional
• Permissions : elasticfilesystem:ClientRootAccess, elasticfilesystem:ClientWrite, elasticfilesystem:ClientMount